52 research outputs found
Hybrid Open-set Segmentation with Synthetic Negative Data
Open-set segmentation is often conceived by complementing closed-set
classification with anomaly detection. Existing dense anomaly detectors operate
either through generative modelling of regular training data or by
discriminating with respect to negative training data. These two approaches
optimize different objectives and therefore exhibit different failure modes.
Consequently, we propose the first dense hybrid anomaly score that fuses
generative and discriminative cues. The proposed score can be efficiently
implemented by upgrading any semantic segmentation model with
translation-equivariant estimates of data likelihood and dataset posterior. Our
design is a remarkably good fit for efficient inference on large images due to
negligible computational overhead over the closed-set baseline. The resulting
dense hybrid open-set models require negative training images that can be
sampled either from an auxiliary negative dataset or from a jointly trained
generative model. We evaluate our contributions on benchmarks for dense anomaly
detection and open-set segmentation of traffic scenes. The experiments reveal
strong open-set performance in spite of negligible computational overhead
Dynamic loss balancing and sequential enhancement for road-safety assessment and traffic scene classification
Road-safety inspection is an indispensable instrument for reducing
road-accident fatalities contributed to road infrastructure. Recent work
formalizes road-safety assessment in terms of carefully selected risk factors
that are also known as road-safety attributes. In current practice, these
attributes are manually annotated in geo-referenced monocular video for each
road segment. We propose to reduce dependency on tedious human labor by
automating recognition with a two-stage neural architecture. The first stage
predicts more than forty road-safety attributes by observing a local
spatio-temporal context. Our design leverages an efficient convolutional
pipeline, which benefits from pre-training on semantic segmentation of street
scenes. The second stage enhances predictions through sequential integration
across a larger temporal window. Our design leverages per-attribute instances
of a lightweight bidirectional LSTM architecture. Both stages alleviate extreme
class imbalance by incorporating a multi-task variant of recall-based dynamic
loss weighting. We perform experiments on the iRAP-BH dataset, which involves
fully labeled geo-referenced video along 2,300 km of public roads in Bosnia and
Herzegovina. We also validate our approach by comparing it with the related
work on two road-scene classification datasets from the literature: Honda
Scenes and FM3m. Experimental evaluation confirms the value of our
contributions on all three datasets.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
On Advantages of Mask-level Recognition for Outlier-aware Segmentation
Most dense recognition approaches bring a separate decision in each
particular pixel. These approaches deliver competitive performance in usual
closed-set setups. However, important applications in the wild typically
require strong performance in presence of outliers. We show that this demanding
setup greatly benefit from mask-level predictions, even in the case of
non-finetuned baseline models. Moreover, we propose an alternative formulation
of dense recognition uncertainty that effectively reduces false positive
responses at semantic borders. The proposed formulation produces a further
improvement over a very strong baseline and sets the new state of the art in
outlier-aware semantic segmentation with and without training on negative data.
Our contributions also lead to performance improvement in a recent panoptic
setup. In-depth experiments confirm that our approach succeeds due to implicit
aggregation of pixel-level cues into mask-level predictions.Comment: Accepted to CVPR 2023 workshop on Visual Anomaly and Novelty
Detection (VAND
- …